Target recognizing device and target recognizing method

ABSTRACT

A target recognizing device that creates a template concerning a target by itself is provided. The device comprises a robot camera for creating continuous images showing a target and an arithmetic operation unit for controlling the robot camera. A robot camera acquires continuous image information by means of a camera rotatable by a motor. The arithmetic operation unit comprises a template management section for allowing an update part to update a template stored in a template storage part and a target recognition section having an image storage part where the acquired image information is stored, a recognizing part for recognizing a target by using the template, and a motion command part for rotating a camera according to the result of recognition.

TECHNICAL FIELD

[0001] The present invention relates to a target recognizing device anda target recognizing method for recognizing a target by using templates.

[0002] This application is based on a patent application in Japan(Japanese Patent Application No. 2001-042299), and the contents in theapplication in Japan are hereby incorporated as a part of thisspecification.

BACKGROUND ART

[0003] In conventional image recognizing devices, various sensors suchas a camera are used to obtain a scene including a target, being anobject to be recognized, as image information, and target recognition isconducted based on the image information and fragmentary knowledge withrespect to the target. The main object of the target recognition is tograsp the scene including the target from the image information asaccurately as possible, and rebuild the target. At this time, as amethod for specifying the target, a template matching method isfrequently used, which obtains a consistency between the target in theimage information, and many templates prepared beforehand with respectto the target.

[0004] In such a conventional target recognizing device however, it isnecessary to prepare many templates so that consistency with varioustargets can be realized, in order to correspond to various conditions.In other words, there is a problem in that before conducting therecognition processing, it is necessary to store the templates for thenumber of types of target, being an object to be recognized, beforehandin the target recognizing device.

[0005] When it is attempted to realize the development of anautonomously movable robot having a device which recognizes a sceneincluding a target, and an autonomously shift device, with aconventional technique, the following method can be considered. That isto say, at first, a plurality of image information relating to a targetpoint is stored beforehand as templates. Image information including thetarget point is then obtained by a camera or the like mounted therein,to extract an area having the best consistency with the template storedin advance, from the image information. The robot then moves toward thedirection corresponding to the extracted area, by the autonomously shiftdevice.

[0006] In dynamic scene analysis for analyzing the movement of a targetobject, by using a plurality of images continuous timewise, an opticalflow is frequently used. The optical flow is a vector representation ofthe direction of movement at one point on the image and the magnitude ofthe velocity thereof. By measuring the optical flow of the targetobject, the movement of the target object can be recognized.

[0007] In the target recognizing device in the autonomously movablerobot however, the scene photographed by a camera or the like changesmoment by moment due to the movement of the robot, and even in the caseof the same target, the size of the target occupying the obtained imageinformation changes. Therefore, even by simply using the optical flow,it is difficult to perform accurate target recognition, and manytemplates corresponding to the respective situations become necessary.

[0008] As described above, in the conventional technique, there is aproblem in that it may be difficult to perform accurate recognitionprocessing of a target in a scene, which is changing timewise.

DISCLOSURE OF INVENTION

[0009] It is an object of the present invention to provide a targetrecognizing device and a target recognizing method, which canself-generate a template relating to the newest target, withoutrequiring storage of a large amount of templates, in which the types,direction and size of the target are changed.

[0010] The first aspect of the present invention provides a devicecomprising a template storing device which stores a template forrecognizing a target, an image acquiring device which acquirescontinuous images including the target, a recognition processing devicewhich detects an optical flow between at least two images of thecontinuous images to obtain an evaluation function value based on theoptical flow; and an update device which updates the template stored inthe template storing device, based on the image including the targetacquired by the image acquiring device, until the evaluation functionvalue exceeds a predetermined value.

[0011] In the above configuration, continuous images including thetarget are acquired, an optical flow between at least two images of theacquired continuous images is detected, and the template is updatedbased on the image including the target, until the evaluation functionvalue based on the optical flow exceeds the predetermined value. As aresult, it is not necessary to store beforehand a large amount oftemplates in which the types, direction and size of the target arechange, and a template relating to the newest target can beself-generated. Moreover, since the template is updated based on theoptical flow obtained from between the respective images, a noisecomponent other than the target can be removed. As a result, thesituation can be prevented where a noise component is included in thetemplate.

[0012] According to the second aspect of the present invention, thedevice further comprises a driving device which rotates the imageacquiring device, and an operation instruction device which outputs anoperation instruction to the driving device based on the instructionfrom the recognition processing device, and the operation instructiondevice outputs an operation instruction for stopping the rotationalmotion when the evaluation function value exceeds the predeterminedvalue.

[0013] In the above configuration, the image acquiring device is rotatedbased on the instruction from the recognition processing device, andwhen a predetermined condition is satisfied, the rotational motion isstopped. As a result, the target recognizing device can performpredetermined operations such as rotating the image acquiring device ina direction of a target recognized by using templates, and shift theimage acquiring device, and hence can be preferably used for controllinga robot.

[0014] According to the third aspect of the present invention, thedevice further comprises: a control signal receiving device whichreceives a control signal indicating a timing at which the template isupdated; and an identification device which identifies the content ofthe control signal received by the control signal receiving device andinforms the update device of the content, and the update device updatesthe template corresponding to an identification result of the controlsignal.

[0015] In the above configuration, since the control signal indicatingthe timing at which the template is updated is received, and thetemplate is updated at the timing when the control signal is received,update of the template for recognizing the target can be performedaccording to an instruction from outside, and when the image acquiringdevice turns to an optional direction, this direction can be designatedas a target direction. Moreover, since the update processing of thetemplate for recognizing the target can be performed from the imageobtained only when the image acquiring device turns to a predeterminedtarget direction, more reliable template generation can be performed.

[0016] According to the fourth aspect of the present invention, theevaluation function value is obtained by comparing the degree ofapproximation of the template and the image including the target,setting a shift quantity with respect to the approximating pixel to theposition in the template, and summing up shift quantities for all pixelsin the whole templates.

[0017] In the above configuration, the degree of approximation of theimage including the target and the template is compared, the shiftquantity of the approximating pixel to the target pixel in the templateis set, and the shift quantities for all pixels in the whole templatesare summed up, to obtain the evaluation function value. As a result,only a characteristic point of the target is emphasized (extracted), andin the target direction, a portion, which is not stably detected, suchas a characteristic point of an object existing irregularly, can beremoved.

[0018] According to the fifth aspect of the present invention, there areprovided at least two-types of templates for use at the time of updatingwhile rotating in one direction, one of which is added with apositive-polarity weighting value, and another one of which is addedwith a negative-polarity weighting value.

[0019] In the above configuration, as the template updated at the timeof rotating in the same direction, at least two templates respectivelyadded with a positive-polarity and a negative-polarity weighting valueare provided. As a result, the robustness with respect to the opticalflow detected in the actual environment can be improved.

[0020] According to the sixth aspect of the present invention, aplurality of template groups is provided depending on the rotationdirection of the image acquiring device.

[0021] In the above configuration, since the template group to beupdated is different, depending on the rotation direction of the imageacquiring device, imbalance of movement in the target direction withrespect to the rotation direction of the image acquiring device can beeliminated, and the detection accuracy of the target can be improved.

[0022] The seventh aspect of the present invention provides a targetidentification method comprises a template storing step in which atemplate for recognizing a target is stored, an image acquiring step inwhich continuous images including the target are acquired byself-rotational motion; a recognition processing step in which anoptical flow between at least two images of the continuous images isdetected, to obtain an evaluation function value based on the opticalflow, and an update step in which the template stored by the templatestoring step is updated based on an image including the target acquiredby the image acquiring step, by template update timing instructed fromoutside, until the evaluation function value exceeds a predeterminedvalue.

[0023] The eighth aspect of the present invention provides a computerprogram for target identification method to be executed by a computer,the program comprises the steps of template storage processing forstoring a template for recognizing a target, image acquisitionprocessing for acquiring continuous images including the target,recognition processing for detecting an optical flow between at leasttwo images of the continuous images, to obtain an evaluation functionvalue based on the optical flow, and update processing for updating thetemplate stored by the template storage processing, based on an imageincluding the target acquired by the image acquisition processing, untilthe evaluation function value exceeds a predetermined value.

[0024] According to the ninth aspect of the present invention, the imageacquisition program to be executed in a computer comprises the steps ofdriving processing for performing self-rotational motion, and operationinstruction processing for outputting operation instructions withrespect to the driving processing, based on instructions from therecognition processing, are further executed by the computer, and theoperation instruction processing outputs an operation instruction forstopping rotational motion at a point in time when the evaluationfunction value exceeds a predetermined value.

[0025] According to the tenth aspect of the present invention, thecomputer program for executing the target identification method in acomputer comprises the steps of control signal reception processing forreceiving a control signal indicating a timing at which the template isupdated, and identification processing for identifying the content ofthe control signal received by the control signal reception processingand informing the content with respect to the update processing arefurther executed by the computer, wherein the update processing updatesthe template corresponding to an identification result of the controlsignal.

[0026] According to the eleventh aspect of the present invention, thetarget identification program comprises the steps of comparing thedegree of approximation of the template and the image including thetarget, setting a shift quantity of the approximating pixel to thetarget position, and summing up the shift quantities for all pixelscorresponding to all pixels of the templates.

[0027] According to the twelfth aspect the present invention, aplurality of templates are provided, which differ depending on therotation direction of the image acquiring device.

[0028] According to the thirteenth aspect the present invention, amongthe plurality of templates to be updated at the time of rotating in thesame direction, there are at least two types of templates, one of whichis added with a positive-polarity weighting value and another one ofwhich is added with a negative-polarity weighting value.

BRIEF DESCRIPTION OF DRAWINGS

[0029]FIG. 1 is a block diagram showing the configuration of oneembodiment of the present invention.

[0030]FIG. 2 is a flowchart showing the operation of a targetrecognizing device shown in FIG. 1.

[0031]FIG. 3 is a flowchart showing the operation of update processingfor templates and weighting shown in FIG. 2.

[0032]FIG. 4 is a diagram showing the operation of a camera 11.

[0033]FIG. 5 is a diagram showing constitution of a template.

[0034]FIG. 6 is a diagram showing the constitution of the degree ofapproximation FIG. 7 is a diagram showing the constitution of a localshift quantity.

[0035]FIG. 8 is a diagram showing the update operation for templates andweighting values.

[0036]FIG. 9 is a diagram showing tracks of a camera 11 in anembodiment.

[0037]FIG. 10 is a diagram showing a track of the direction of thecamera 11, when the target recognition processing is executed by usingsufficient templates for recognizing the target in the above embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

[0038] The target recognizing device according to one embodiment of thepresent invention will be described, with reference to the drawings.FIG. 1 is a block diagram showing the configuration of this embodiment.Reference symbol 1 denotes a robot camera comprising a camera 11 (forexample, a CCD camera) for acquiring peripheral images, and a motor 12for rotating the camera 11 in the horizontal direction to direct theoptical axis of the camera 11 to an optional direction. The motor 12 isprovided with an encoder (not shown), so that the direction (angle ofrotation) of the camera 11 at the time of rotational motion can bedetected by a value of the encoder. Reference symbol 2 denotes anarithmetic operation section, which comprises a target recognitionsection 21 and a template control section 22. Reference symbol 23denotes an image storage section for storing images taken by the camera11, and images stored herein are sampled and quantized images. Referencesymbol 24 denotes a recognition processing section for recognizing atarget based on the image stored in the image storage section 23.Reference symbol 25 denotes an operation instruction section foroutputting operation instructions to the robot camera 1, based on thetarget recognition result in the recognition processing section 24.Reference symbol 26 denotes a control signal reception section forreceiving a control signal Rn transmitted from outside or from the robotcamera 1. Reference symbol 27 denotes an identification section foridentifying the control signal Rn and issuing an instruction based onthe identification result. Reference symbol 28 denotes an update sectionfor updating a template stored in a template storage section 29.

[0039] The outline of the operation of the target recognizing devicewill be described, with reference to FIG. 1. n pieces (n is a naturalnumber) of templates Mn are stored in advance in the template storagesection 29 in the template control section 22.

[0040] The camera 11 provided in the robot camera 1 takes images of theexternal environment as seen from the robot camera 1 continuously, andthe image information is sequentially stored in the image storagesection 23 in the target recognition section 21. The recognitionprocessing section 24 in the target recognition section 21 calculatesthe optical flow between the continuous images taken out from the imagestorage section 23, and changes in the operation are obtained for eachlocal portion in the image to extract a characteristic portion in theimage. The recognition processing section 24 searches the template Mncorresponding to the input image in the template storage section 29, toperform recognition processing of a target in the external environment.The processing result in the recognition processing section 24 isprovided to the operation instruction section 25. Upon reception of theprocessing result, the operation instruction section 25 provides anoperation instruction corresponding to the processing result in therecognition processing section 24, to the motor 12 in the robot camera1. The motor 12 generates a driving force corresponding to the operationinstruction, to thereby cause spontaneous rotational motion of thecamera 11.

[0041] On the other hand, the templates Mn stored in the templatestorage section 29 are updated as required. Update of the templates Mnis performed corresponding to a control signal Rn provided from outsideof the target recognizing device, or a control signal Rn transmittedfrom the robot camera 1 at the timing when the camera turns to adirection including the target. The control signal Rn is received by thecontrol signal reception section 26, and the content thereof isidentified by the identification section 27. The identification section27 provides an update instruction corresponding to the identificationresult, to the update section 28. The update section 28 updates thecorresponding template Mn based on the received update instruction.

[0042] The operation of the target recognizing device will be describedin detail, with reference to FIGS. 2 and 3. FIG. 2 is a flowchartshowing the operation for creating the template Mn corresponding to theimage including a target. FIG. 3 is a flowchart showing the operation instep S22 shown in FIG. 2. At first, the operation instruction section 25issues an instruction for performing rotational motion, to the motor 12.Upon reception of the instruction, the motor 12 drives, to rotate thecamera 11 about an axis perpendicular to the ground.

[0043] The rotation operation of the camera 11 will be described here.FIG. 4 is a diagram of the robot camera 1 as seen from the above. Thoughthe computation section 2 is not shown in FIG. 4, the computationsection 2 is installed in the vicinity of the robot camera 1. As shownin this figure, the camera 11 performs reciprocating rotational motionwithin the range of rotational motion of from −50° to 50°, due todriving of the motor 12. At this time, before starting the rotationalmotion, the direction to which the optical axis of the camera 11 isdirected is assumed to be 0°. This direction of 0° is the directionwhere the target exists (target direction). In this processing, theimage in the target direction is acquired, and a template correspondingto the acquired image is created. In this embodiment, the camera 11 isrotated to obtain the optical flow from, the image in the vicinity ofthe target direction and the image in the target direction, which isthen used for creating and updating the template.

[0044] When the image in the target direction and the image in thevicinity of the target direction are acquired, it can also be consideredto simply rotate the camera in the vicinity of the target direction, butin order to create a better template relating to the image in thevicinity of the target direction, it is necessary to take pictures ofthe image repetitively. Therefore, in order to obtain stable images atthe time of rotational motion of the camera in the vicinity of thetarget direction, the range of rotational motion is designated as up to50° right and left. The encoder provided in the motor 12 outputs “1” asthe control signal Rn, when the angle of rotation is 0°. This controlsignal Rn is received by the control signal reception section 26. Byreceiving this control signal Rn, the computation section 2 can detectthat the camera 11 has turned to the target direction, and creates andupdates the template Mn, by using the image at this timing.

[0045] In FIG. 4, the camera 11 acquires an image in the direction of 0°(target direction) indicated by reference symbol S. The camera 11 thenrotates to the direction of −50° and returns to the direction of 0°. Atthis time, the camera 11 acquires images at a certain sampling interval.Further, the camera 11 rotates to the direction of +50° and returns tothe direction of 0°. Also at this time, the camera 11 acquires images ata certain sampling interval. The optical flow is calculated from aseries of continuous images obtained here.

[0046] The images acquired by the camera 11 are sequentially stored inthe image storage section in the computation section 2 (step S21 in FIG.2). The stored image is described here as an image It(x, y). Each pixelin the image It(x, y) is expressed by a gray value of 8 bits (256gradations).

[0047] In parallel with the operation for storing the images, the updatesection 28 updates the template Mn and the weighting value Wn (stepS22). Details of the processing in step S22 will be described later.

[0048] The recognition processing section 24 then calculates the degreeof approximation Sn(x, y) between the newest image It(x, y) stored inthe image storage section 23 and each pixel in the n templates Mn heldin the recognition processing section 24 beforehand (step S23). Theoperation for calculating the degree of approximation Sn(x, y) will bedescribed here, with reference to FIGS. 5 and 6. FIG. 5 is a diagramshowing one template Mn representing the templates Mn, which arepreviously stored. FIG. 6 is a diagram showing the degree ofapproximation Sn for each pixel obtained by the processing in step S23.

[0049] As shown in FIG. 5, each template Mn is constituted of 20×20pieces of pixels, and the position of each pixel is expressed by thecoordinates (x, y). The gray value of each pixel in all templates Mn inthe initial state is set to any value of 256 gradations (0 to 255) atrandom. The construction may be such that the template storage section29 stores a transform value for each of the 20×20 pixels, by using aFourier transform, a Gabor transform or a Wavelet transform, instead ofthe gray value.

[0050] The template Mn can correspond to an optional position on theinput image It acquired by the camera 11. For example, if the pixelIt(0, 0) at the upper left of the input image It is made to agree withthe pixel Mn(0, 0) at the upper left of the template Mn, the imagerecognition for 20×20 pixels at the upper left of the input image It iscarried out, by using the template Mn.

[0051] In the embodiment described later, an experiment was carried outfor setting such that the center of the input image It agrees with thecenter of the template Mn, but the template Mn may be used for the otherareas of the input image It.

[0052] The degree of approximation Sn can be obtained by performing foreach pixel, processing where the newest image It(x, y) stored in theimage storage section 23 is compared with the template Mn(x, y) storedin the template storage section 29 for each pixel to obtain a differencein the gray value, and if the difference is smaller than a predeterminedvalue, it is assumed that the image It(x, y) approximates to thetemplate Mn(x, y), and if the difference is larger than thepredetermined value, it is assumed that the image It(x, y) does notapproximate to the template Mn(x, y). In other words, a difference Dn(x,y) between the gray value Mn(x, y) of the template and the gray valueIt(x, y) of the input image is calculated as follows:

Dn(x, y)=|Mn(x, y)−It(x, y)|  (1).

[0053] If Dn(x, y) is smaller than the predetermined threshold Gn, it isassumed that the degree of approximation Sn=1, and if Dn(x, y) is notsmaller than Gn, it is assumed that the degree of approximation Sn=0. Bythis processing, as a result of comparison of the input image It withthe template Mn, “1” is set to only the approximating pixels.

[0054] The recognition processing section 24 calculates the optical flowby using an input image It−1(x, y) obtained immediately before thenewest input image It(x, y), and the input image It(x, y), to obtain theoptical flow Vx (step S24). The optical flow is calculated by thefollowing equation (2):

∂xVx(x, y)+∂yVy(x, y)+∂t=0  (2).

[0055] Since the camera 11 performs only the right and left rotationalmotion, then in the optical flow, only the x component is used, andVx(x, y) obtained by the equation (2) is designated as the optical flow.

[0056] The recognition processing section 24 calculates a local shiftquantity Vn from the obtained optical flow Vx(x, y), using the followingequation (3) (step S25):

Vn(x, y)=Sn(x, y)·Vx(x, y)  (3).

[0057] This reflects the optical flow value, with respect to only thepixel having the degree of approximation Sn=1. The result of theobtained local shift quantity Vn is shown in FIG. 7.

[0058] The recognition processing section 24 calculates the evaluationfunction value B by the following equation (4) (step S26):$\begin{matrix}{B = {\sum\limits_{n}^{\quad}{\alpha \quad n{\sum\limits_{x}^{\quad}{\sum\limits_{y}^{\quad}{{{Wn}\left( {x,y} \right)}{{Vn}\left( {x,y} \right)}}}}}}} & (4)\end{matrix}$

[0059] where, Wn(x, y) is a weighting value with respect to the pixel,which can take a positive or negative value, and αn is a weighting valuefor each template in a range of 0<αn≦1. This Wn(x, y) is a valueobtained in step S22. The processing in step S26 means to sum up valuesincluding the respective weighting portions for all pixels of alltemplates. In other words, with an increase in the number of pixelshaving a large weighting value Wn and a degree of approximation Sn=1,the evaluation function value B increases.

[0060] The recognition processing section 24 determines whether theabsolute value of the obtained evaluation function value B is largerthan a predetermined threshold k (step S27). As a result of thisdetermination, if the evaluation function value B is not larger than thethreshold k, control returns to step S21 and step S22, to repeat theprocessing. On the other hand, if the evaluation function value B islarger than the threshold k, the recognition processing section 24informs the operation instruction section 25 that the template creationprocessing has finished. In response to this, the operation instructionsection 25 issues an instruction for stopping the drive of the motor 12to the motor 12. As a result, the rotational motion of the camera 11stops, with the camera 11 turning to the target direction (step S28).

[0061] As a result, the template Mn corresponding to the image in thetarget direction is complete, thereby enabling the target recognitionprocessing using this template Mn.

[0062] If the threshold k is set to a very low value, the evaluationfunction value B is likely to exceed the threshold k. Therefore, thecamera 11 can be quickly located at a position, which is considered tobe the target direction. However, if the threshold k is set to a valuetoo low, there is the possibility that the camera 11 is locatederroneously in a direction where image information similar to the targetdirection is obtained. On the contrary, if the threshold k is set to avalue too high, the camera 11 is reliably located in the true targetdirection, but it takes too much time. Moreover, if the rotational speedof the camera 11 is too fast as compared with the number of pixels inthe template Mn, since the change in the evaluation function value Bcorresponding to the rotation of the camera 11 becomes large, there isthe possibility that the camera 11 is stopped with the camera 11 turningto an inadequate direction.

[0063] Therefore, it is desired to appropriately set the threshold kcorresponding to various parameters, such as the complexity of theenvironment where the robot camera 1 is placed, the recognition accuracyrequired for the target recognizing device and the time required forrecognition, the processing capability of the computation section 2, thenumber of pixels in the template Mn and the number of templates Mn, andthe rotational speed of the camera 11.

[0064] Moreover, when |B|>k is not satisfied, even if the processing offrom steps S21 to S27 is repeated, after the processing is performed fora predetermined number of times, or after a predetermined period oftime, the largest value of |B| may be detected, so that the camera 11 isstopped, directed in that direction.

[0065] The update processing for the template Mn and the weighting valueWn shown in step S22 in FIG. 2 will be described, with reference toFIGS. 3 and 8. FIG. 3 is a flowchart showing the operation of the updateprocessing for the template Mn and the weighting value Wn. At first, theidentification section 27 determines whether the control signal Rn is“1” (ON) (step S31). This control signal Rn is such a signal that whenthe camera 11 turns to the target direction (the direction of 0° shownin FIG. 4), “1” is output. As a result of the determination, if thecontrol signal Rn is not “1”, it is determined whether |Vn|>d (stepS34), wherein d denotes a preset threshold. As a result of thedetermination, if |Vn| is not larger than d, the control signal Rn “1”has not been input, and the camera 11 should not turn to the targetdirection. Therefore, the update processing for the template and thelike is not executed, and the processing finishes. If |Vn|>d, the updatesection 28 sets the weighting value Wn to “0” (step S35). Thisprocessing means that a pixel having the size of Vn larger than thepredetermined value, though the control signal Rn “1” has not beeninput, is regarded as noise as a result of having caught an object otherthan the target, and the record of this portion is cancelled (deleted)by designating the weighting value Wn as “0”, so that the updateprocessing for the template and the like is not executed.

[0066] On the other hand, in step S31, when the control signal Rn is“1”, the update section 28 determines whether |Vn|>d, and Wn>0 (stepS32). As a result of the determination, if |Vn|>d, and Wn>0, the updatesection 28 updates the template Mn and the weighting value Wn (stepS33), to finish the processing. The respective update is performedaccording to the following equations (5) and (6).

Wn(x, y)=Wn(x, y)+w  (5)

[0067] where w denotes an initial value of the weighting value.

Mn(x, y)=Mn−q(Mn−It)  (6)

[0068] where q denotes a fixed value set to any figure in the range of0<q≦1

[0069] In other words, as the pixel in the template Mn becomes closer tothe input image It, the initial value w is added to the weighting valueWn to increase the weighting, so that the template Mn is brought closeto the gray value of the pixel in the input image It. As a result, whilethe camera 11 performs the rotational motion repetitively, thecharacteristic point of the target is emphasized (extracted) in theinput image It, and the portion which is not detected stably in thetarget direction, such as a characteristic point of an object existingirregularly, is cut out.

[0070] On the other hand, in step S32, if |Vn| is not larger than d, orWn is not larger than 0, the update section 28 determines whether theweighting value Wn is “0” (step S36). As a result of the determination,if Wn is not 0, the processing finishes. On the other hand, when Wn=0,the update section 28 sets the template Mn and the weighting value Wn(step S37). This processing is performed even if the target may havebeen caught, but since the weighting value Wn is still 0, it isdetermined that the image of this portion is obtained for the firsttime, and Wn is set to w (initial value), and the pixel is set to a grayvalue of a corresponding pixel in the input image It.

[0071]FIG. 8 is a diagram for explaining the update processing for thetemplate Mn and the weighting value Wn. In FIG. 8, there is shown a casewhere the templates Mn are four (M1 to M4). The control signal Rn isprovided for updating the template Mn of a corresponding sign. Forexample, when a control signal R1=1 is given, the template M1 isupdated.

[0072] In FIG. 8, patterns 1 and 3 show the state where the camera 11 isnot directed to the target direction (direction of 0°). At this time,the control signal Rn is R1=R2=R3=R4=0, and corresponds to a case of“NO” in step S31 shown in FIG. 3. Patterns 2 and 4 show the state wherethe camera 11 is directed to the target direction (direction of 0°). Atthis time, the control signal Rn is R1=R2=1, or R3=R4=1, and correspondsto a case of “YES” in step S31 shown in FIG. 3.

[0073] In pattern 2 in FIG. 8, the templates M1 and M2 are updated, butweighting values W1 and W2 with polarities opposite to each other areprovided. Similarly, in pattern 4, the templates M3 and M4 are updated,but weighting values W3 and W4 with polarities opposite to each otherare provided. This is for improving the robustness with respect to theoptical flow by means of an object detected in the actual environment.For example, in the actual environment, the boundary portion of anobject is likely to be detected as an optical flow in a directionopposite to that of the face portion, and the detection result isunstable. Therefore, in order to improve the detection accuracy as awhole, weighting values Wn having opposite polarities are used toclarify the boundary portion.

[0074] As shown in FIG. 8, the updated templates Mn are grouped (into M1and M2, or M3 and M4) according to a case where the camera 11 reachesthe direction of 0° (target direction) counterclockwise (pattern 2), anda case where the camera 11 reaches the direction of 0° (targetdirection) clockwise (pattern 4). This is for coping with the imbalanceof the movement in the target direction with respect to the movement ofthe camera 11. Normally, it is rare that an object existing in thetarget direction has a completely symmetric shape or pattern, and theway of generating the optical flow is different in the counterclockwiserotation and the clockwise rotation. In the present invention, in orderto correspond to the imbalance and to improve the detection accuracy,the templates Mn are grouped for each direction of rotation. In otherwords, the characteristic of the optical flow generated by an objectexisting in the target direction, when the camera 11 is rotatedcounterclockwise, is reflected in the templates M1 and M2, and thecharacteristic of the optical flow generated by the object existing inthe target direction, when the camera 11 is rotated clockwise, isreflected in the templates M3 and M4.

[0075] The camera 11 may be set so as to continue to rotate in eitherone direction. In this case, a target recognizing device needs only torecord and hold the templates M1 and M2, or the templates M3 and M4.

[0076] When the camera 111 is to be stopped, there is the possibilitythat the camera 11 is stopped in the state of being displaced too muchand shifted from the target direction or the target position due to theinfluence of inertia. Therefore, the construction may be such that astopping instruction is issued before the target direction (0°) by apredetermined angle, or the camera 11 is made to return by apredetermined angle and stop reliably, with the camera facing the targetdirection. Moreover, the angle of rotation of the camera 11 is notlimited to ±50°, and other angles may be employed. The update processingfor the templates Mn and the like may be performed at an angle otherthan 0°, for example, at a point in time when the camera 11 turns to adirection where a pixel or pattern having a particular density exists.

[0077] By the above described processing operation, the template usedfor recognizing a predetermined target is complete, and stored in thetemplate storage section 29.

[0078] The results of tests conducted for preparing the template by theabove described operation will now be described. FIG. 9 is a diagramshowing tracks of the direction of the camera 11. As shown in FIG. 9, inthe first test, when in total four control signals Rn were provided, theabove described at-end condition was satisfied and the test finished. Inother words, it is shown that when the fourth control signal Rn isprovided, the template Mn(x, y) having had a random value becomes asufficient value for recognizing the target in the direction of 0°.

[0079] The second test was conducted by using the template Mn(x, y)obtained as a result of the first test. In the second test, when thecamera 11 was turned from the direction of +50° to the direction of 0°,as in the first test, the control signal Rn was provided. In this case,the test finished by providing two control signals Rn. In other words,the initial value was different from that of the first test,corresponding to the target. As a result, it is seen that the number ofcontrol signals Rn provided for generating the template used for thetarget recognition can be reduced.

[0080] In the third test, the template obtained as a result of thesecond test was used, to start the test from the state where the camera11 was directed in the direction of −50°. In this case, the at-endcondition was satisfied by only providing a first control signal. Inother words, it is seen that the template required for recognizing thetarget has been nearly completed from the initial stage.

[0081] As shown in FIG. 9, it is seen that by repeating the templateupdate processing, the template is updated to one sufficient forrecognizing the target, and the camera is located in the vicinity of 0°in the target direction at an early stage.

[0082] The test results of the target recognition processing using thecomplete template in the target recognizing device will be described.FIG. 10 is a diagram showing a track of the direction of the camera 11,when the target recognition processing is performed by using a templatesufficient for recognizing the target, obtained by repeating thetemplate update processing. As shown in FIG. 10, it is seen that even ifthe direction to which the camera 11 is made to turn at first iscounterclockwise, or clockwise, the target is recognized quickly insubstantially the same time, and the camera 11 is located in the targetdirection.

[0083] The robot camera 1 and the computation section 2 may be connectedby wire, but a radio communication apparatus may be provided, and theinformation exchanged by using radio communication. The robot camera 1and the computation section 2 may be integrated.

[0084] Furthermore, when a control signal Rn is transmitted from outsideto the control signal reception section 26, the output timing of thecontrol signal Rn may be determined by an operator, every time thecamera 11 turns to the target direction.

[0085] Moreover, a program for realizing the function of each blockshown in FIG. 1 may be recorded in a computer readable recording medium,and the program recorded in the recording medium may be read into acomputer system, and executed, to conduct the template update processingand the target recognition processing. The “computer system” referred toherein includes OS and hardware such as peripheral equipment. The“computer readable recording medium” stands for portable media such asflexible disks, magneto-optical disks, ROM and CD-ROM, and memories suchas hard disks built into the computer system. The “computer readablerecording medium” further includes one holding the program for a certainperiod of time, such as a volatile memory (RAM) within the computersystem, which becomes a server or a client, when the program istransmitted via a network such as the Internet or a communicationcircuit such as telephone lines.

[0086] The program may be transmitted from the computer system, whichstores the program in a memory or the like, to other computer systemsvia a transmission medium, or by transmitted waves in the transmissionmedium. The “transmission medium” for transmitting the program standsfor a medium having a function of transmitting the information like anetwork (communication network) such as the Internet, or a communicationcircuit (communication line) such as telephone lines. The program may befor realizing a part of the above described function. Moreover, theprogram may be a so-called differential file (differential program)which can realize the above described function by a combination with aprogram already recorded in the computer system.

[0087] As described above, the target recognizing device of the presentinvention comprises: a template storing device which stores a templatefor recognizing a target; an image acquiring device which acquirescontinuous images including the target; a recognition processing devicewhich detects an optical flow between at least two images of thecontinuous images, to obtain an evaluation function value based on theoptical flow; and an update device which updates the template stored inthe template storing device, based on the image including the targetacquired by the image acquiring device, until the evaluation functionvalue exceeds a predetermined value. As a result, it is not necessary tostore a large amount of templates beforehand, in which the types,direction and size of targets are changed, and a template relating tothe newest target can be self-generated.

[0088] The target recognizing device of the present invention canperform predetermined operations, such as rotating the camera in adirection of a target recognized by using the template, and shift thecamera. Hence, the target recognizing device of the present invention issuitable for controlling a robot.

1. A target recognizing device comprising: a template storing devicewhich stores a template for recognizing a target; an image acquiringdevice which acquires continuous images including said target; arecognition processing device which detects an optical flow between atleast two images of said continuous images, to obtain an evaluationfunction value based on said optical flow; and an update device whichupdates said template stored in said template storing device, based onthe image including said target acquired by said image acquiring device,until said evaluation function value exceeds a predetermined value.
 2. Atarget recognizing device according to claim 1, further comprising; adriving device which rotates said image acquiring device, and anoperation instruction device which outputs an operation instruction tosaid driving device based on the instruction from said recognitionprocessing device, wherein said operation instruction device outputs anoperation instruction for stopping the rotational motion when saidevaluation function value exceeds a predetermined value.
 3. A targetrecognizing device according to claim 2 further comprising: a controlsignal receiving device which receives a control signal indicating atiming at which said template is updated; and an identification devicewhich identifies the content of the control signal received by saidcontrol signal receiving device and informs said update device of thecontent, wherein said update device updates said template correspondingto an identification result of said control signal.
 4. A targetrecognizing device according to claim 3, wherein said recognitionprocessing device obtains said evaluation function value by comparingthe degree of approximation of said template and the image includingsaid target, setting a shift quantity with respect to the approximatingpixel corresponding to the image including said target, and summing upthe shift quantities for all pixels in the whole templates.
 5. A targetrecognizing device according to claim 4, wherein the target recognizingdevice comprises a group of templates for images acquired while rotatingthe image acquiring device, and the group of template is updatedaccording to the rotation direction of said image acquiring device.
 6. Atarget recognizing device according to claim 5, wherein said group oftemplates which are updated while rotating in the same directioncomprises at least two types of templates, one of which is added with apositive-polarity weighting value, and the other one of which is addedwith a negative-polarity weighting value.
 7. A target recognizing methodcomprising: a template storing step in which a template for recognizinga target is stored; an image acquiring step in which continuous imagesincluding said target are acquired by self-rotational motion; arecognition processing step in which an optical flow between at leasttwo images of said continuous images is detected in order to obtain anevaluation function value based on said optical flow; and an update stepin which said template stored by said template storing step is updated,based on an image including said target acquired by said image acquiringstep, by template update timing instructed from outside, until saidevaluation function value exceeds a predetermined value.
 8. A targetrecognition program for executing in a computer comprising theprocessing steps of: storing a template in an template storing devicefor recognizing a target; acquiring continuous images including saidtarget through a image acquiring device; detecting an optical flowbetween at least two images of said continuous images and obtaining anevaluation function value based on said optical flow; and updating saidtemplate stored in said template storing device based on an imageincluding said target acquired by said image acquiring device, untilsaid evaluation function value exceeds a predetermined value.
 9. Atarget recognition program for executing in a computer according toclaim 8, said processing step of acquiring continuous images comprisingthe processing steps of: outputting an instruction for driving a drivingdevice of an image acquiring device to perform spontaneous rotationmotion when an instruction is output from said target recognitiondevice; and outputting an instruction for stopping the rotational motionof the image acquiring device when said evaluation function valueexceeds a predetermined value.
 10. A target recognition program forexecuting in a computer according to claim 9, wherein the programfurther comprising the processing steps of: receiving a control signalindicating a timing at which said template is updated, and identifyingthe content of the control signal received in said processing step ofreceiving the control signal and informing the content of the controlsignal to the update device; wherein said update device updates saidtemplate according to the result of the processing step of identifying nprocessing of said control signal.
 11. A target recognizing programaccording to claim 10, wherein said processing step of recognizing thetarget further comprising the steps of: comparing a degree ofapproximation of said template and the image including said target;setting shift quantities for pixels of the approximating target to apixel of the target in the template; and summing up shift quantities forall pixels in the whole templates for obtaining said evaluation functionvalue.
 12. A target recognition program according to claim 11, wherein aplurality of template groups are provided for use in updating templatesaccording to the rotation directions at the time of image acquiringprocessing.
 13. A target recognition program according to claim 12,wherein said plurality of templates to be updated at the time ofrotating in the same direction include at least two templates, one ofwhich is added with a positive-polarity weighting value and the otherone of which is added with a negative-polarity weighting value.